Classification and Volume Morphing for Performance-driven Facial Animation

We present a new facial animation approach that produces a wide variety of realistic facial expressions with advantages over existing geometry deformation and morphing methods for performance-driven animation systems. These systems sense and reconstruct a real person's facial expressions from an image sequence. The novelty of our approach is to sense and animate both textures and geometry using classification, volume morphing, and 3D-texture interpolations. Feature tracking drives our volume morphing animation system directly, without a need for intermediate animation parameters. Classification encodes realistic dynamic skin wrinkles, eye blinking, and eye motions. Classification results control 3D-texture animations that reconstruct the observed facial appearance. Classification leads to extremely low bandwidths for communicating visual information and independence of the rendered resolution and image quality from sensing resolution and pose. We demonstrate our approach with results obtained with a realtime volume morphing engine that animates the geometry of a facial model from sparse motion samples. A wavelet-based classifier and a 3D-texture engine reproduce dynamic textures on the animated geometry, including wrinkling effects, eye blinking, and eye movements.